Multi-body Depth-Map Fusion with Non-intersection Constraints

Depthmap fusion is the problem of computing dense 3D reconstructions from a set of depthmaps. Whereas this problem has received a lot of attention for purely rigid scenes, there is remarkably little prior work for dense reconstructions of scenes consisting of several moving rigid bodies or parts. This paper therefore explores this multi-body depthmap fusion problem. A first observation in the multi-body setting is that when treated naively, ghosting artifacts will emerge, ie. the same part will be reconstructed multiple times at different positions. We therefore introduce non-intersection constraints which resolve these issues: at any point in time, a point in space can only be occupied by at most one part. Interestingly enough, these constraints can be expressed as linear inequalities and as such define a convex set. We therefore propose to phrase the multi-body depthmap fusion problem in a convex voxel labeling framework. Experimental evaluation shows that our approach succeeds in computing artifact-free dense reconstructions of the individual parts with a minimal overhead due to the non-intersection constraints.